I'm having exactly the same issue with a Raspberry Pi Zero and a raphnet 4nes4snes adapter (using 2 NES controllers). The controllers show up when plugged in, regardless of how they are plugged in, but only work when plugged into a hub that is plugged into the RP0. On occasion the direct-plugged controller works but I cannot repeat this.
I think there is some issue with how the raphnet adapter generates the USB signals.